Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement support for encoding HDR10+ from JSON metadata file #3000

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Implement support for encoding HDR10+ from JSON metadata file #3000

wants to merge 1 commit into from

Conversation

quietvoid
Copy link

@quietvoid quietvoid commented Aug 25, 2022 •
edited
Loading

The goal of this PR is to allow users to encode HDR video with HDR10+ metadata, in compliance with the AV1 HDR10+ specification.

In the current state, the changes make it possible to encode HDR10+ for both CLI and Rust API usage.

From CLI, the metadata is expected to be passed as a JSON metadata file, which follows the same format as encoders like x265.
The metadata parsing/encoding is done through the hdr10plus crate.
The library is probably less than ideal but it should be fine to parse things at init.

The CLI opt can be either --hdr10plus-json or --dhdr10-info (used by x265)

From Rust, the metadata must be encoded into the final T.35 and provided for the frames that require it.

Output bitstream validated with the AOM AV1 HDR10+ Validator.

Starting this as a draft, as some things need to be improved;

  • Behaviour when some parsed metadata is invalid.
    • Either the encoder has to abort, or keeps going with an incomplete map of frame<->metadata.
  • Need to make the ST2094-40 prefix const.
  • Improve comments.
  • ... probably more, feedback welcome.

lu-zero
lu-zero reviewed Aug 25, 2022
View reviewed changes
src/api/config/encoder.rs Outdated Show resolved Hide resolved
@quietvoid
Copy link
Author

quietvoid commented Aug 25, 2022 •
edited
Loading

As is, this is probably ready for review.
I'm just not sure how the CLI encoder should behave when the JSON metadata partially fails to parse.

Also, hopefully it also works in rav1e-ch, as long as the frame numbers are continuous across GOPs.
edit: rav1e-ch seems to work just fine.

@quietvoid quietvoid marked this pull request as ready for review August 25, 2022 12:47
@quietvoid
Copy link
Author

quietvoid commented Aug 25, 2022 •
edited
Loading

It does appear that the current T.35 insertion code skips over some frames with show_frame=0 and show_existing_frame=1.
At least compared to the reference stream in https://github.com/AOMediaCodec/av1-hdr10plus/wiki, which has the metadata OBUs for every frame.

Though the spec (draft) does say

HDR10+ Metadata OBUs are not provided when show_frame=0.

So I'm not sure if the ref sample is correct.
cc @tdaede

There is also an updated figure of the structure: https://raw.githubusercontent.com/AOMediaCodec/av1-hdr10plus/0e2a5d82ec9b7c24e3c4bf84c31557eb99563dd1/obu_tu.png

@codecov-commenter
Copy link

codecov-commenter commented Aug 26, 2022 •
edited by codecov bot
Loading

Codecov Report

Patch coverage: 40.78% and project coverage change: -1.78% ⚠️

Comparison is base (184bf4b) 88.10% compared to head (ee4d6d1) 86.32%.

❗ Current head ee4d6d1 differs from pull request most recent head 82dd0df. Consider uploading reports for the commit 82dd0df to get more accurate results

Additional details and impacted files 
@@            Coverage Diff             @@
##           master    #3000      +/-   ##
==========================================
- Coverage   88.10%   86.32%   -1.78%     
==========================================
  Files          85       89       +4     
  Lines       33596    34122     +526     
==========================================
- Hits        29599    29457     -142     
- Misses       3997     4665     +668
Files Changed Coverage Δ
src/api/util.rs 59.45% <ø> (+0.79%) ⬆️
src/bin/common.rs 51.37% <6.89%> (-1.68%) ⬇️
src/api/internal.rs 96.21% <59.09%> (-1.57%) ⬇️
src/api/config/encoder.rs 89.36% <100.00%> (ø)
src/api/test.rs 99.23% <100.00%> (-0.38%) ⬇️

... and 64 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@quietvoid quietvoid marked this pull request as draft September 19, 2022 13:55
@quietvoid quietvoid force-pushed the t35_hdr10plus branch 2 times, most recently from 003623b to 0627800 Compare January 18, 2023 06:22
@quietvoid
Copy link
Author

As the specification seems to be in a final state, I've reworked things to work properly for the OBU placement.
https://aomediacodec.github.io/av1-hdr10plus/#placement

The T.35 metadata is added at the end when present in the EncoderConfig hdr10plus_payloads.
And only if the existing frame T.35 metadata doesn't already contain one for HDR10+ metadata.

@quietvoid quietvoid marked this pull request as ready for review January 18, 2023 06:26
quietvoid
quietvoid commented Jan 18, 2023
View reviewed changes
src/encoder.rs Outdated Show resolved Hide resolved
@quietvoid quietvoid mentioned this pull request Jan 18, 2023
Open
@lu-zero lu-zero requested a review from tdaede January 18, 2023 14:36
@lu-zero
Copy link
Collaborator

lu-zero commented Jan 18, 2023

clippy seems unhappy, I let @tdaede confirm the actual specification details.

@quietvoid
Copy link
Author

Doh. I was running clippy with --all-features and it was failing.

@quietvoid quietvoid mentioned this pull request Aug 7, 2023
Open
@quietvoid
Copy link
Author

quietvoid commented Aug 8, 2023 •
edited
Loading

It was pointed out on the IRC that the payloads should probably be handled on the CLI side only, as the API EncoderConfig already has T.35 metadata support.
However the current T.35 API implementation has issues with missing metadata that I've yet to resolve, so I'll update this when the issues are worked out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lu-zero lu-zero Awaiting requested review from lu-zero

@tdaede tdaede Awaiting requested review from tdaede

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@quietvoid @codecov-commenter @lu-zero